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Classical Fourier analysis has an exact counterpart in group theory and in some areas of 
geometry. Here I'll describe how this goes for nilpotent Lie groups, for a class of Riemannian 
manifolds closely related to a nilpotent Lie group structure. There are also some infinite 
dimensional analogs but I won't go into that here. The analytic ideas are not so different 
from the classical Fourier transform and Fourier inversion theories in one real variable. 

Here I'll give a few brief indications of this beautiful topic. References, proofs and related 
topics for the finite dimensional theory can be found a recent AMS Monograph/Survey 
volume pQ . If you are interested in the infinite dimensional theory you may also wish to look 
at the article [2] appear soon in Mathematische Annalen. 

In Section [1] I'll recall a few basic facts on classical Fourier theory and note the connec- 
tion with the theory of locally compact abelian groups and their unitary representations. 
In Section [2] we look at the first noncommutative locally compact groups, the Heisenberg 
groups H n . We describe their unitary representations, Fourier transform theory and Fourier 
inversion formula. 

The coadjoint orbit picture is the best way to understand representations of nilpotent 
Lie groups. It is guided by the example of the Heisenberg group. We indicate that theory in 
Section [3j Then in Section [4] we come to a class of connected, simply connected, nilpotent Lie 
groups with many of the good analytic properties of vector groups and Heisenberg groups. 
Those are the simply connected, nilpotent Lie groups with square integrable representations. 

In Section we push some of the analysis to a class of homogeneous spaces where the 
techniques and results are analogous to those of locally compact abelian groups. Those are 
the commutative spaces G/K, i.e. the Gelfand pairs (G, K). We have already had a glimpse 
of this in Section [2] for the semidirect products H n x K and the riemannian homogeneous 
spaces (H n x K)/K where K is a compact group of automorphisms of H n . In Section [6] 
we look more generally at Fourier transform theory and the Fourier inversion formula for 
commutative nilmanifolds (N x K) / K where N is a simply connected, nilpotent Lie groups 
with square integrable representations and K is a compact group of automorphisms of N. 

As indicated earlier, Fourier analysis for nilpotent Lie groups iV and commutative nil- 
manifolds (N x K)/K, where N has square integrable representations, has recently been 
extended to some classes direct limit groups and spaces. 
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1 Classical Fourier Series 



Let's recall the Fourier series development for a function / of one variable that is periodic of 
period 2-n. One views / as a function on the circle S = {e tx }. The circle S is a multiplicative 
group and we expand / in terms of the unitary characters 

Xn '■ S — > S by Xn(x) = e mx , continuous group homomorphism. 

Then the Fourier inversion formula is 

oo 

f( x ) = Yl /( n )x« 

n=— oo 

where the Fourier transform 

f(n) = l f(x)e- mx dx=(f,xn)msy 
Jo 

The point is that / is a linear combination of the Xn with coefficients given by the Fourier 
transform /. This uses the topological group structure and the rotation-invariant measure 
on S. 

One has a similar situation when the compact group S is replaced by a finite dimensional 
real vector space V. Let V* denote its linear dual space. If / e L l {V) fl L 2 {V) the Fourier 
inversion formula is 

/(*) = (i) m/2 1 m)e**dz 

Jy 

where the Fourier transform is 

RO = (IT 1 ' 2 [ f(x)e-*°*dx=(f,xt)»<y ) . 
Jv 

Again, / is a linear combination (this time it is a continuous linear combination) of the 
unitary characters Xt( x ) = ^ 011 V ■> an( i ^ ne coefficients of the linear combination are given 
by the Fourier transform /. 

It is the same story for locally compact abelian groups G. The unitary characters form 
a group 

G = {x '■ G — ?■ S continuous homomorphisms} 

with composition (xiX2)(x) = Xi( x )X2(x)- It is locally compact with the weak topology for 
the evaluation maps ev x : x ^ x( x )- If / £ ^(G) fl L 2 (G) the Fourier inversion formula is 

f( x ) = / f(x)x( x )dx 
Jg 
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where the Fourier transform 



Kx) = / f(x)x(x)dx = (/,x)l2 (g) . 
Jg 

As in the Euclidean cases, Fourier inversion expresses the function / as a (possibly con- 
tinuous) linear combination of unitary characters on G, where the coefficients of the linear 
combination are given by the Fourier transform. 

In this context, />—>■/ preserves L 2 norm and extends by continuity to an isometry 
of L 2 {G) onto L 2 {G). In effect this expresses L 2 {G) as a (possibly continuous) sum of G- 
modules, 

L 2 (G) = / C x dx where C x is spanned by x- 
Jg 

In this direct integral decomposition dx could be replaced by any equivalent measure, so 
that decomposition is not as precise as the Fourier inversion formula. 



2 The Heisenberg group 

Next, we see what happens when we weaken the commutativity condition. The first case of 
that is the case of the Heisenberg group. There the Fourier transform and Fourier inversion 
are in some sense the same as in the classical case of a vector group, except that some of the 
integration occurs in the character formula and the rest in integration over the unitary dual. 

The Heisenberg group of real dimension 2m + 1 is 

H m = Im C + C m with group law (z, w) {z\ w') — (z + z' + Im (w, w'),w + w') 

where Im denotes imaginary component (as opposed to the coefficient of \/— T), z,z' G Z := 
ImC and w,w' G W := C m ). Its Lie algebra, the Heisenberg algebra, is 

fj m = 3 + tt> = Im C + C m with [z + w,z' + w'] = (z + z' + 2 Im (w, w')) + (w + w'). 

Here Z = exp(j) is both the center and the derived group of H m , and its complement 
W = exp(ro) ^ R 2m . 

Unitary characters have to annihilate the derived group of H m , in other words factor 
through H m /Z, so the only functions that can be expanded in unitary characters are the 
functions that are lifted from H m /Z. Thus we have to consider something more general. 

The space H m of (equivalence classes of) irreducible unitary representations of H m breaks 
into two parts, one consisting of the 1-dimensional representations and the other of the 
infinite dimensional representation. This goes as follows. 



3 



• One-dimensional representations. These are the ones that annihilate the center Z, and 
are given by the unitary characters x& £ G W^*, on VU = M 2m . 

• Infinite dimensional representations. These are the tt^ = Ind ^ m (xc) where 

N = ImC + R m C # m and C G 3* \ {0} . 

Recall the definition of the induced representation 7r^ = lnd^ ! m (xc)- Here Xc extends from 
Z to A" by xc( z i w ) = Xc( z )- Thus we have a unitary line bundle over H m /N associated to 
the principal iV-bundle H m — > H m /N by the action w : t (->■ Xc( w )^ of AT on C. Now 7T£ is 
the natural action of if m on the space of L 2 sections of that line bundle. 

The classical "Uniqueness of the Heisenberg commutation relations" says that ( deter- 
mines the class [n^\ G H m . And restriction to Z shows that [ir^] = [n^] just when ( = 

Using the fact that ( determines [ir^\ , one realizes [tt^\ by an action of H m on the Hilbert 
space 1-L m of Hermite polynomials on C m . The maximal compact subgroup of Aut(H m ) is 
the unitary group U(m). Its action is 

g : (z,w) H> (z,g(w)). 

Result: extends to an irreducible unitary representation ttq of the semidirect product 
H m x U(m) on T-L m . So if K is any closed subgroup of U(m) then tt^h^k is an irreducible 
unitary representation of H m x K on H m - The Mackey Little Group theory says that 
H^aK = {[tT c ®fi]|[fi]6i and C G 3* \ {0}}. 

3 Representations and Coadjoint Orbits 

Kirillov theory for connected simply connected nilpotent Lie groups A" realizes their unitary 
representations in terms of the the coadjoint representation of N, that is, the representation 
ad* of A" on the linear dual space n* of its Lie algebra n. 

On the group level the coadjoint representation is given by (Ad*(n)/)(£) = /(Ad(n) _1 £). 
Write Of for the (coadjoint) orbit Ad* (AT)/ of the linear functional /. Consider the anti- 
symmetric bilinear form bf on n* given by bf(^,i]) = f{[^,i]}). The kernel of bf is the Lie 
algebra of the isotropy subgroup of AT at /. Thus bf defines an Ad*(A")-invariant symplectic 
form Uf on the coadjoint orbit Of. The symplectic homogeneous space (Of,ujf) leads to a 
unitary representation class [717] G N, as follows. 

Let Nf denote the Ad*(A r )-stabilizer of /. Its Lie algebra x\f is the annihilator of /, 
in other words ti/ = {y G n | f(v, n) = 0}. A (real) polarization for / is a subalgebra 
pen that contains ti/, has dimension given by dim(p/ti/) = |dim(n/n/), and satisfies 
/([p,p]) = 0. Under the differential n — > Tf(Of) of A" — > Of, real polarizations for / are in 
one to one correspondence with AMnvariant integrable Lagrangian distributions on (Of, uf). 
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Fix a real polarization p for / and let P = exp(p). It is the analytic subgroup of N for 
p and it is a closed, connected, simply connected subgroup of N. In particular e % f : P — > C 
is a well defined unitary character. That defines a unitary representation 

*■/ = *■/,„ = bid ?(e") 

of N. The basic facts are given by 

Theorem 3.1 Let N be a connected simply connected nilpotent Lie group and f En*. 

1. There exist real polarizations p for f . 

2. If p is a real polarization for f then the unitary representation nf t p is irreducible. 

3. If p and p' are real polarizations for f then the unitary representations iif^ and 7Ty jP / 
are equivalent, so the class \jTf] £ N is well defined. 

4. If [it] G N then there exists h G n* such that [it] = [tth]- 

In other words, / h-> 7T/ iP induces a one to one map of n* / Ad* (N) onto N. 

To see just how this works, consider the case where N is the Heisenberg group H m , and 
let / G T)* m . Here the center 3 = ImC and its complement = C n . Decompose = u + ro 
where u = W 1 and ro = iM. n . Note Im (u, u) = = Im (to, ro). If /(j) = we have the real 
polarization p = f) m . If /(g) 7^ we have the real polarization p = 3 + u. That demonstrates 
Theorem 13. IT 1 ) . 

If /(3) = then 7T/ iP is a unitary character on if n , automatically irreducible. Now 
suppose 7(3) 7^ and p = 3 + u. Then 7Tj iP is a representation of H n on L 2 {G/ P) = L 2 (W) 
where W = exp(ro) = W 1 . Then TTf^{H n ) acts by all translations on W and by scaling that 
distinguishes the integrands of L 2 (W) = J mt e'^'C d£, so it is irreducible. That demonstrates 
Theorem 13.1( 2) . 

If f($) = then \) n is the only real polarization for /. Now suppose f(i) 7^ and 
consider the case where p = 3 + u and p' = 3 + tv. Then u = Im h(-, ■) pairs u with ro, 
and the Fourier transform J 7 : L 2 (U) = L 2 (W) intertwines 7rj p with ftp p. More generally, 
if p and p' are any two real polarizations for /, then we write p = (p fl p') + u', p' = 
(p fl p') + ro' and p fl p' = 3 + 1)'. That done, cu pairs u' with ro', and the corresponding Fourier 
transform J 7 : L 2 (U') = L 2 (W) combines with the identity transformation of L 2 (V) to give 
a map L 2 (V')®L 2 (U') = L 2 (V')®L 2 (W') that intertwines 7T/ )P with 7T//, p . This demonstrates 
Theorem 13.1( 3). 

Now Theorem 13.1( 4) follows from the considerations we outlined in Section [2j In the 
terminology there, the infinite dimensional irreducible unitary representation ttq of H n is 
equivalent to 7Tj whenever / G n* such that f(z) = ((,x) for every z G 3. In particular, if 
f(d) where 3 is the center of the Heisenberg algebra, then the coadjoint orbit Of = /+3" 1 , 
where ^ := {h G n* | h(i) = 0}. Of course one can also verify this by direct computation. 
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4 Square Integrable Representations 



In this section N is a connected simply connected nilpotent Lie group and Z is its center. If 
(eZwe denote — {[k] £ N \ iz\z is a multiple of (. The corresponding L 2 space is 



(4.2) L 2 (N/Z :Q:= If :N measurable 



f(nz) = ((z) 1 f(n) and 

I N/ z\f( n )\ 2d ^N /Z ( nZ ) < 



OO 



The inner product (/, h)^ = f N , z f{n)h{n)d(i Nfz {nZ) is well defined on the relative L 2 space 

L 2 (N/Z : (). Each iV c is a measurable subset of iV, and N = |J ?e z N c . Here L 2 (N) = 
fgL 2 (N/Z : (). This decomposes the left regular representation of N as 

£ = Indf 1} (l) =Indflndf 1} (l) = Indf [ ( d( = /lndfC^C= / k d C 

J z J z J z 

where = Ind^C is the left regular representation of N on L 2 (N/Z : Q. The corresponding 
expansion for functions, 



f( n ) = / /c( n ) rf C where / c (n) = / f(nz)((z)dfi z (z), 
J z J z 

is just Fourier inversion on the commutative locally compact group Z. 

Now we describe some results of Moore and myself on square integrable representations 
in this context. The first observation is 



Theorem 4.3 Let N be a connected simply connected nilpotent Lie group and £ £ Z . If 
[k] £ N$ then the following conditions are equivalent. 

1. There exist nonzero u,v £ such that \ f UjV \ £ L 2 (N/Z), i.e., f U}V £ L 2 (N/Z : £). 

2. T/ie coefficient \ f u ,v\ £ L 2 (N/Z), equivalently f UjV £ L 2 (N/Z : £), /or a// -u,t> £ i^. 

3. [7r] «s a discrete summand of Iq. 



A representation class [7r] £ iV is L 2 or square integrable or relative discrete series 

if its coefficients f u ,v( n ) — fw.u,v( n ) '■= (u,ir(n)v) satisfy \f u>v \ £ L 2 (N/Z), in other words 
if its coefficients are square integrable modulo Z. Theorem 14.31 says that it is sufficient to 
check this for just one nonzero coefficient, and Theorem 14.3( 3) justifies the term "relative 
discrete series". 

We say that iV has square integrable representations if at least one class [tt] £ N 
is square integrable. These representations satisfy an analog of the Schur orthogonality 
relations: 
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Theorem 4.4 Let N be a connected simply connected nilpotent Lie group. If C, G Z and 
[it] G N{ is square integrable then there is a number deg(7r) > such that the coefficients of 
tt satisfy 



(4.5) 



/«! lVl (n) f U2 ,v 2 (n) dfj, (nZ) 



i 



(ui,u 2 ) (vi,v 2 ) 



N/Z 



deg(7r) 



for all Ui,Vi G H n . If [tti], [7r 2 ] G are inequivalent square integrable representations then 
their coefficients are orthogonal in L 2 (N/Z : (), 



for all Ui, V\ G H 7T1 and m 2 , t> 2 G H^ 2 . 

The number deg(7r) is the formal degree of [tt]. It plays the same role in Theorem 14.41 
as that played by the degree in the Schur orthogonality relations for compact groups . In 
general, deg(7r) depends on normalization of Haar measure: a rescaling of Haar measure fi N/z 
of N/Z to cfx N/z rescales formal degrees deg(7r) to \ deg(7r). We don't see this for compact 
groups because there we always scale Haar measure to total mass 1. 

Theorems 14.31 and 14.41 only require that N be a locally compact group of Type I and 
that Z be a closed subgroup of the center of N. They can be understood as special cases of 
Hilbert algebra theory. Here we related them to the Kirillov theory. 

Given / G n* we have the bilinear form bf(x,y) = f([x,y]), the coadjoint orbit Of = 
Ad* (AT)/, the associated representation [717], and the character ( G Z such that [717] G N^. 
Note that f\ } determines the affine subspace / + ^ in n*. 

Theorem 4.7 Let N be a connected simply connected nilpotent Lie group and f G n*. Then 
the following conditions are equivalent. 

1. [717] is square integrable. 

2. The left regular representation £^ of N on L 2 (N/Z : £) is primary. 

3. Of = f +3 , determined by the restriction f\ } . 

4. bf is nondegenerate on n/3. 

Recall the notion of the Pfaffian Pf(cu) of an antisymmetric bilinear form u on a finite 
dimensional real vector space V relative to a volume form v on V. If dim V is odd then by 
definition Pf(cu) = 0. If dimV^ = 2m even, then u m is a multiple of v, and by definition 
that multiple of Pf(u>); in other words u m = Pf(u)v. The Pfaffian is the square root of the 
determinant on antisymmetric bilinear forms. 



(4.6) 
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Fix a volume element v on := n/3. If / G n* we view Uf(x,y) = f([x, y]) as an 
antisymmetric bilinear form on 0. Define P(/) := Pf(u;/). Then P is a homogeneous poly- 
nomial function on n*, and P(f) depends only on /| 3 a So there is a homogeneous polynomial 
function (which we also denote P) on 3* such that P(/) = P(f\ } ). 

In the case of the Heisenberg group H m , P is the homogeneous polynomial P(C) = C(zo) m 
of degree m on 3*. Here the choice of nonzero z Q G 3 is a normalization, in effect a choice of 
unit vector. As described in Theorem 14.101 below, this also gives the formal degree of [71/] 
where ( = f[ r Further, as described in Theorem I4.12[ it gives the Plancherel measure on 

H ra- 
in view of Theorem 14.71 we now have 

Theorem 4.8 The representation Hf is square integrable if and only if the Pfaffian polyno- 
mial P(f\ } ) 7^ 0. In particular <fi : /| 3 1— >■ [71"/] defines a bisection from {A G 3* | P(A) 7^ 0} 
onto {[tt] G N I [tt] is square integrable}. 

One can view the polynomial P as an element of the symmetric algebra 5(3), and since 
3 is commutative that symmetric algebra is the same as the universal enveloping algebra 3- 
From this one can prove 

Corollary 4.9 The group N has square integrable representations if and only if the inclusion 
3 ^ of universal enveloping algebras, induced by 3 c — >• n, mops 3 onto i/ie center of%l. 

Both formal degree and the polynomial P are scaled by 1/c when Haar measure on N/Z 
is scaled by c, so the following is independent of normalization of Haar measure on N/Z. 

Theorem 4.10 The formal degree of a square integrable representation [itf] = <f>(f\ s ) is 
given by deg(vr / ) = 

As in the semisimple case, the infinitesimal character of a representation class [tt] G N 
is the associative algebra homomorphism ^ : Ce?ii(9T) — > C from the center of the enveloping 
algebra, such that dTr(Q is scalar multiplication by Xw(0- (Initially this holds only on C°° 
vectors, but they are dense in H.,,, so by continuity it holds on all vectors.) If ( G 3 then 
Xtt/(C) = */(0- Now, from Theorem 14. 101 

Corollary 4.11 If [tt] G N then the formal degree deg(7r) = |x„-(P)| where we understand 
the formal degree of a non square integrable representation to be zero. 

For the Plancherel formula and Fourier inversion we must normalize Haar measures. 
Choose Haar measures /i z and H N/Z \ they define a Haar measure fi N by dfi N = dfi N/z d/i z , 
i.e. 

/ f(n)dfi N (n)= / ( / f(nz)dfi z (z))dfi (nZ). 
J N Jn/z \Jz J 
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Now we have Lebesgue measures v z , v N/z and v N on 3, n/3 and n specified by the condition 
that the exponential map have Jacobian 1 at 0, and they satisfy dv N = di> N/z dv z . Normalize 
Lebesgue measures on the dual spaces by the condition that Fourier transform is an isometry; 
that gives Lebesgue measures v* , v* and v* such that dv* = dv* dv* . 

& & Z' N/Z N N N/Z Z 

Theorem 4.12 Let N have square integrable representations. Let c = m\2 m where 2m is 
the maximum dimension of the Ad* (N) -orbits in n*. Then Plancherel measure for N is 
concentrated on the square integrable classes and its image under the map 

(f)' 1 : {[71] e N I [71-] is square integrable} ->■ {A e f \ P{\) ^ 0} 

of Theorem 14.81 is c\P(x)\dv*(x) . 

5 Commutative Spaces — Generalities 

We just described the Fourier transform and Fourier inversion formulae for H m — and a 
somewhat larger class of connected simply connected nilpotent Lie groups. Now we edge 
toward a more geometric setting, that of commutative spaces, which is a common general- 
ization of Riemannian symmetric spaces, locally compact abelian groups and homogeneous 
graphs. It is interesting and precise for the cases that involve connected simply connected 
nilpotent Lie groups with square integrable representations. 

A commutative space G/K, equivalently a Gelfand pair (G, K), consists of a locally com- 
pact group G and a compact subgroup K such that the convolution algebra L 1 (K\G/K) is 
commutative. There are several other formulations. Specifically, the following are equivalent. 

1. (G,K) is a Gelfand pair, i.e. L 1 (K\G/K) is commutative under convolution. 

2. If g,g' G G then fi KgK * ^ Kg , K = ^ Kg , K * f^KgK (convolution of measures on K\G/K). 

3. C C (K\G I K) is commutative under convolution. 

4. The measure algebra Ai(K\G / K) is commutative. 

5. The (left regular) representation of G on L 2 (G/K) is multiplicity free. 

If G is a connected Lie group one can add 

6. The algebra V(G,K) of G-invariant differential operators on G/K is commutative. 

Commutative spaces G/K are important for a number of reasons. First, they are manage- 
able because their basic harmonic analysis is very similar to that of locally compact abelian 
groups. We will describe that in a moment. Second, in the Lie group cases, most of the G/K 
carry invariant weakly symmetric Riemannian metrics, which have properties very similar 
to those of Riemannian symmetric spaces. Third, the invariant differential operators and 
corresponding spherical functions play a definite role in special function theory. And fourth, 
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in the nilpotent Lie groups setting, there is some interesting interplay between geometry and 
hypoellipticity. 

We only consider the basic harmonic analysis, here in general and in Section [6] for the 
case of commutative nilmanifolds. 

Analysis on locally compact abelian groups is based on decomposition of functions in 
terms of unitary characters. In the classical euclidean case these are just the complex ex- 
ponentials x$ '■ V ~ C, £ G V*, given by xs. — e lx '^. For a commutative space G/K the 
appropriate replacements are the positive definite spherical functions, defined as follows. 

A continuous ii"-bi-invariant function ip : G — > C is K- spherical if (p(l) = 1 and / h- > 
(/ * i s a homomorphism C C (K\G/K) — > C. Equivalent: (p is not identically zero, and 

if gi, gi G G then 

^OiM^) = / v(gikg 2 )dk. 

J K 

A function <p : G — > C is positive definite if X] vGtT" 9i)cj c i = whenever {ci, . . . , c n } C C 
and {5-1, . . . ,#„} C G. 

Denote V = V(G,K): positive definite i^-spherical functions on G. There is a one-one 
correspondence ip <-¥ 7r v between V and the irreducible unitary representations n of G that 
have a .fT-fixed unit vector v. It is given by (p(g) = (v,n(g)v)y, n - We have the spherical 
transform 

S : / ^ / from L l {K\G/K) to functions on P 

defined by 

S(/)foO = fiip) = (/ * p)(l) = / f{g)y{g- l )dg. 

Jg 

The corresponding spherical inversion formula is 

f(g) = / ?{v)v{g)dn{v)- 

Jv 

Here V has natural structure of locally compact space and /1 is called Plancherel measure. 
The spherical transform 

S : L\K\G/K) n L 2 {K\G/K) L 2 (P,//) 

preserves L 2 norm and extends by continuity to an isometry 

S : L 2 {K\G/K) = L 2 {V,n). 

Note that 5 can only be given by its defining integral expression when that integral converges. 
This is why it has to be extended by L 2 continuity. Of course this problem is already present 
with the classical Fourier transform on M. 
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The Plancherel Formula S : L 2 (K\G/K) = L 2 (V,fi) gives a continuous direct sum 
(direct integral) decomposition 

L 2 (K\G/K) = [ Cy? dfi(<p). 

This extends to a continuous direct sum decomposition 

L\GjK) = [ d»(<p). 
Jv 

Of course all this depends on knowledge of the Plancherel measure /i. 

6 Commutative Nilmanifolds 

Theorem of Carcano (special case): Let K C U(m) acting on C m , where f) m = ImC + C m 
with center ImC. Then (H m x K)/K is commutative if and only if the representation of 
K c , on the ring of all polynomials on C m , is multiplicity free. 



Kac classified the connected K m that are irreducible on C 





Group K m 


Acting on 




Group K m 


Acting on 


1 


SU(m) 


C m , m ^ 2 


8 


U(n) 


C m = A 2 (C") 


2 


U(m) 


C m , m ^ 1 


9 


SU{t) x SU(n) 


C m = C* ® C", 


3 


Sp(n) 


C m = C 2 ™ 


10 


S(U(£) x U{n)) 


C m = C £ ® C" 


4 


U(l) x Sp(n) 


C m = C 2« 


11 


17(2) x Sp(n) 


C m = C 2 ® C 2 " 


5a 


U(l) x SO(2n) 


C m = C 2 ™ 


12 


SI7(3) x 5p(n) 


C m = C 3 ® C 2 " 


5b 


U(l) x SO(2n+l) 


(£*2n-\-± 


13 


[7(3) x Sp(n) 


C m = C 3 ® C 2 " 


6 


U(n),n ^ 2 




14 


SI7(n) x S P (4) 


<C m = C" ® C 8 


7 


SU(n),n odd 


C m = A 2 (C") 


15 


U(n) x 5p(4) 


<C m = C n ® C 8 



A commutative nilmanij 'old is a commutative space G/i^ where some connected closed 
nilpotent subgroup of G is transitive on G/K. 

Example: G/K where G = H m x K m and K = K m , where K m occurs in the table above. 

Fact: Let G/K be commutative. If a conn closed nilpotent subgroup of G is transitive 
then is the nilradical of G, N is abelian or 2-step nilpotent, and G = N xi K . 

In particular: Commutative nilmanifolds have form G/K where G/K = (N xi K)/K, N 
is not so different from the Heisenberg group and K C Aut(A^). 

More examples: a commutative nilmanifold (G = N xi K)/K is irreducible if K acts 
irreducibly on n/[n,n], maximal if it is not of the form (G/Z,K/Z) with {1} ^ Z d K 
central in G. They have been classified by Vinberg. Let n = 3 + to where 3 is the center and 
Ad(K)w = to. Then 3 is the center of n, ro = n/[n, n] as 7^-module, and the classification is 
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Group K 


ro 


3 


1 


SO(n) 


E" 


SkcwK™ XI1 =so(n) 


2 


Spin{l) 


K 8 = 


K 7 = Im 


3 


G 2 


R Y = Im0 


E' = Ini0 


4 


U{l)-SO(n),n^A 


C™ 


ImC 


5 


(U(l)-)SU(n) 


C n 


A 2 C" 8 ImC 


6 


SU(n),n odd 


C™ 


A 2 C n 


7 


SU(n), n odd 


C n 


ImC 


8 


U(n) 


C" 


ImC" x " = u(n) 


9 


(U(l)-)Sp(n) 


H" 


ReM" xn ffilmi 


10 


U{n) 


S 2 C™ 


E 


11 


{U(l)-)SU(n),n ^ 3 


A 2 C" 


E 


12 


U(l) ■ Spin(7) 


C 8 


R 7 ®R 


13 


U(l) ■ Spin(9) 




R 


14 


{U(l)-)Spin(10) 


c ib 


R 


15 


U(l)-G 2 


C 


R 


16 


U(l)-E 6 


C 2y 


R 


17 


Sp(l) x Sp(n),n^ 2 


H" 


ImH = sp(l) 


18 


Sp(2) x Sp(n) 


ff x " 


ImH 2x2 = Sp(2) 


19 


{U(l)-)SU{m) x SU(n) 
m, n ^ 3 


C m ® C" 


E 


20 


(U(1)-)SU(2) x SU(n) 


C 2 <g> C" 


ImC 2x2 = u(2) 


21 


(U(l)-)Sp(2) x SU(n),n ^ 3 


H 2 ® C n 


E 


22 


J7(2) x S'p(n) 


C 2 <g> M™ 


ImC 2x2 = u(2) 


23 


U(3) x 5p(n),n ^ 2 


C 3 <g> M™ 


E 



where the optional U(l) is required in (5) when n is odd, in (11) when n is even, in (19) 
when m = n, in (20) when n = 2, and in (21) when n ^ 4. Here (9) was the first known 
case where G/K is not weakly symmetric (Lauret). 

To make this explicit one needs to know the positive definite spherical functions and the 
Plancherel measure. 

In the connected Lie group cases, the K-spherical functions on G are just the joint 
eigenfunctions for V(G, K), and in many cases this is how one finds them. We look at a few 
of those cases. 

Case (G, K) = (R n x K, K) where K is transitive on the unit sphere in JR n . Then the 
invariant differential operators are the polynomials in the Laplacian A = — ^<9 2 /<9xf, and 
the K-spherical functions are the radial eigenfunctions of A for real non-negative eigenvalue. 
They are the 

<P&) = (llel|r)-^ 2)/ V (n „ 2)/2 (||e||r) 

where r = \\x\\ and J u is the Bessel function of first kind and order v. 

Case (G, K) = (H n x U(n),U(n)). In the coordinate (z,w) G ImC + C n = H n the 
invariant differential operators are the polynomials in d/dz, □ = — J29 2 /dw 2 and □. The 
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positive definite spherical functions corresponding to 1-dimensional representations are the 

<Pt(z,w,k) = ^^Jn-MW \\w\\) for + i G to* 
and those for infinite dimensional representations are the 

'e i ^L^- 1 \C(z)\\w\\ 2 )e-^^ 2 / 4 if C(i) > 0, 



VCm ■ (z,W,k) I— >■ 



<p_ c>m (z,iu,A;) if C(«) < 0, 



where ( G (Im C)* and Lm ^ is the generalized Laguerre polynomial of order n—\ normalized 
to L^ 1) (0) = 1. 
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